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ABSTRACT 

We report on our development of an educational social machine based on the concept that feedback in communities is an 
effective means to support the development of communities of learning and practice. Key challenges faced by this work 
are how best to support educational and social interactions, how to deliver personalised tuition, and how to enable 
effective feedback, all in a way which is potentially scalable to thousands of users. A case study is described involving 
one to one and group music lessons in an on-campus, face to face, higher education context that were observed and 
analysed in terms of the actions carried out by the participants. The actions are described and it is shown how they can be 
formalised into a flowchart which represents the social interactions and activities within a lesson. Through this analysis, 
specific scenarios emerged where the feedback being given might not be effective, e.g. the recipient not understanding 
the feedback or the provision of feedback which is not specific enough. In answer to these scenarios of ineffective 
feedback, the requirements for a technological intervention which aims to make the feedback more effective are 
proposed. With this in mind, we are then able to describe a novel technological platform which has been developed as 
part of a large-scale European research project and which aims to support effective feedback. The platform is based 
around focused discussion of time based media, embedded within existing teaching activities at a research led higher 
education institution in the UK. We outline how it is being used in a blended learning model to support the teaching and 
learning of music. We reflect on the experience of developing techniques and systems for enabling communities of e- 
learning and describe our evaluation methodology which involves several, ongoing case studies and approximately 400 
users in its current phase. 
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1. INTRODUCTION 

Our research project is concerned with the development of a social machine which aims to support and 
enhance the experience of learning music through the optimal provision of feedback. Key challenges we face 
in this work are how to support educational and social interactions, how to deliver personalised music tuition, 
and how to enable effective feedback, all in a way which is potentially scalable to thousands of users. 

In this paper, we present our method for addressing these challenges through an initial period of teaching 
observation and analysis followed by the development of a technological platform via a participatory design 
process. The methodology is summarised in figure 1. Following that, two key research outputs are presented: 
an analysis of one to one and group music tuition within our institution and a novel e-learning platform we 
have developed in response to this analysis. The teaching analysis resulted in a list of archetypical teaching 
and learning activities, shown in table 1, an ontology of musical feedback, shown in figure 3 and flowcharts 
describing interactions within lessons as shown in figure 2. The technological platform is essentially a 
repository for audio and video recordings which allows the user to upload media then to share it with 
communities of other users who can then place comments relating to the media along a timeline. It is 
described as a set of system requirements in table 2 and as screenshots of its media discussion interface and 
social timeline in figures 4 and 5. 
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1.1 Background 

Let us first consider what we mean by a social machine. Tim Berners Lee is credited with having coined the 
term social machine in 2000: 

Computers can help if we use them to create abstract social machines on the Web: processes in 

which the people do the creative work and the machine does the administration (Berners-Lee et al., 

2000 ). 

This quote is contextualised in the transition to web 2.0 where the process of publishing content and 
interacting online was democratised with technologies such as blogs, social networks and so on. In 2013, we 
find ourselves in the age of the social machine, where the point of interest for internet technologies is no 
longer the architectural underpinnings but the way in which people and machines interact within these 
systems. De Roure et al., are concerned with the observation of these social machines and provide some 
examples: Wikipedia, Ushahidi, Galaxy Zoo, reCAPTCHA and Mechanical Turk (Roure et al., 2013). 
Moving to the educational context, 2012 was the ‘year of the MOOC’ (Pappano, 2012); indeed, one of the 
authors of this paper ran a MOOC with an enrolled student body of 97,000. With their extreme student to 
staff ratios, MOOCs rely upon interactions between peers for support and assessment; this is a level of social 
interaction that seems beyond what has been seen previously within standard VLEs. Since they are 
technological systems supporting a range of social interactions, we consider them to be another example of a 
social machine. 

Now let us consider the term ‘feedback’. We define feedback in the educational context simply as a 
reaction to a learner’s output which is somehow made visible to the learner. In higher education in general, 
feedback is considered very important. It is one of the key areas covered by the UK National Student Survey 
and historically one of the lower scoring areas in terms of student satisfaction (HEFCE, 2011). So feedback is 
important and is not always being done well, but how can we do it better? Juwah et al. present a list of 7 
principles of good feedback in higher education, wherein good feedback a) Facilitates assessment (reflection) 
in learning b) Encourages teacher and peer dialogue around learning c) Helps clarify what good performance 
is (goals, criteria, expected standards) d) Provides opportunities to close the gap between current and desired 
performance, e) Delivers high quality information to students about their learning f) Encourages positive 
motivational beliefs and self-esteem and g) Provides information to teachers that can be used to help shape 
the teaching (Juwah et al., 2004). These are useful general principles but music education is a specific case 
where the contexts and nature of feedback are perhaps quite different. Therefore, in this paper we will present 
our analysis of feedback within music education with specific examples, then show how we have developed a 
technological system which aims to support that specialised kind of feedback. 

1.2 Previous Work 

In this section, we will provide a brief overview of some related work in the areas of social discussion of 
media, online music education and peer interactions. The platform provides a media repository and timeline 
based discussion functionality; a similar commercial platform is Soundcloud, which allows users to maintain 
and share a repository of audio files and to post comments to a timeline (Bird, 2014). Considering the 
concept of annotations placed on a timeline, Latulipe discusses various projects using timeline based 
discussion systems including the ‘Video Collaboratory’ (Latulipe, 2013). Puig et al. developed the ‘Lignes de 
Temps’ software which provides a multitrack timeline aiming to promote polemical discussion (Puig and 
Monnin, 2006). Moving to the music education area, there are a range of commercial online platforms such 
as ArtistWorks (Marshall et al., 2014) and Berkley Online from the Berklee School of Music. Indeed Berklee 
have been running musical MOOCs on the coursera platform, using SoundCloud for peer discussion 
(Nuernberg and Perrier, 2013). There has also been significant public research undertaken into technology for 
music education, such as the European funded i-maestro and VEMUS projects, both of which focused in part 
on the specificity of feedback (Ong et al., 2006), (Fober et al., 2007). The concept of social interactions 
between students within VLEs did not arrive with the xMOOC in 2012, of course; the cMOOC which came 
before it had perhaps a more radical, distributed pedagogy (Smith and Eng, 2013). Going further back, 
forums have been a standard component in VLEs for a long time and new types of VLEs emphasising social 
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interactions have been reported in the literature. For example, Shi et al. describe their Topolor system which 
enables ‘social personalized adaptive e-learning’ (Shi et al., 2013). Finally, to contextualise our methodology, 
we use a grounded theory approach to analyse our lesson observations and a participatory design approach to 
develop the features of the platform (Charmaz, 2006), (Muller and Kuhn, 1993). 

1.3 Research Questions 

Our research project has several high level research questions: 

(1) How well does our approach increase participation in musical learning activity? 

(2) How important is giving and receiving feedback online for engagement with practice? 

(3) How do we correlate engagement and feedback in a community? 

(4) What is the right level of social coordination and structure that students want for online -supported 
learning? Can we provide interfaces for non-technical people to design social coordination? 

(5) How can we evidence musical competencies and musical development in students? 

(6) How can automatic techniques be used to evidence feedback in music learning? 

In the work presented here, we describe our ‘approach’ and provide evidence about the nature and 
importance of feedback which underpins several of the questions above. We also provide answers to how one 
might evidence musical competencies. 

1.4 Structure of this Paper 

The background and motivation for the work has been presented in this section. In section 2 we will describe 
our methodology for building social machines combining teaching observation and participatory design. In 
section 3 we present the outputs of the methodology including the observed teaching and learning activities, 
types of feedback and a description of the features of our new platform. In section 4 we describe the ongoing 
evaluation of the platform with 400 users. The paper ends with a discussion and conclusion in section 5. 


2. A METHODOLOGY FOR BUILDING SOCIAL MACHINES 

COMBINING TEACHING OBSERVATION AND PARTICIPATORY 
DESIGN 

The development of our platform has taken place in 4 phases. In phase 1, teaching observation, we observed 
and recorded 23 undergraduate instrumental and vocal performance lessons at our institution. The lessons 
involved 9 teachers teaching guitar, voice, piano and group and 14 individual students. The lessons were in 
either one to one or group format and spanned the popular and classical music degrees. Recordings of the 
lessons were transcribed to approximately 500 pages of text and notes were taken by the researcher observing 
the lessons. In phase 2, analysis, a grounded theory approach was used to code the activities within the 
lessons in order to identify key teaching and learning activities. This approach ‘fosters seeing your data in 
fresh ways and exploring your ideas about the data through early analytic writing’ (Charmaz, 2006). The 
activities were then organised into higher level descriptions in the form of flowcharts describing different 
types of lessons. A particular emphasis was placed on the flow of feedback between participants in these 
lesson archetypes. In phase 3, basic requirements, we drew up some basic requirements for the platform in 
order for it to support the teaching effectively. This would allow us to bootstrap the basic functionality of the 
platform ready for the next phase. In that phase, participatory design, we used a participatory design 
approach, where the input of users is sought and acted upon throughout the iterated development lifecycle 
(Muller and Kuhn, 1993). In a sense, the final phase includes its own observation, analysis and requirements 
phases, except that the observations are of users using the system (for real teaching and learning). This final 
phase is ongoing. Figure 1 illustrates the relationship between the 4 phases. 
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Figure 1 . The 4 phases of platform development 


3. OUTPUTS FROM THE METHOD 

In this section we will present the outputs generated by the teaching analysis and participatory design 
process. 

3.1 Enumerating Teaching and Learning Activities 


We were able to identify 9 distinct teaching and learning activities from our lesson transcripts and 
observation notes and these are shown in table 1 with examples of each from the transcripts. 

Table 1. The 9 distinct teaching and learning activities in one to one and group music lessons 


Activity 

Transmission 


Performance modelling 


Identify and solve 


Practicing solutions 


Feeding back 


Checking student 
understanding 


Discussion of goals and 
ideas 


Description 

Tutors provide theoretical 
and practical information to 
students 

Tutors or students perform 
good and bad examples of 
extracts from a composition 1 
Identify, discuss then suggest 
solutions to performance 
problems. 


Students put the solutions 
from the identify and solve 
activity into practice in their 
playing 

Self, peer and tutor feedback 
on a performance, after it has 
happened 

Initiated by student or tutor, 
student understanding is 
verified through dialogue 


Discussion and negotiation of 
assessment or other goals and 


Example 

so whatever you do to your mouth, it's the same sound 
because the tongue is going right up against the soft 
palette, so the sound can only come out in your nose. 

A musical activity 


Okay, did you hear that? The music is very uneven... 
Let's experiment a bit. Let's do it this way. I'll play the 
right hand with you the first time. I am going to go for 
just a legato version. Then you will have a go at it hands 
together and I would like you try to a legato version so 
then you are not affected by the separation of the notes. 
A musical activity 


That's fine, that sounded pretty good. The very first time 
it sounded - your down beat sounded a little bit like v oh 
this is a down beat, I'm going to play loud now.' Always 
be careful about how you're shaping it. 

[Tutor ] From there, just flatten the 3 and you've got 
Dorian and add to that flatten the 6, you've got Aeolian, if 
you want to continue, what would you do next? Anybody 
know? [Student] Flatten the 2? [Tutor] Exactly right! 
Flatten the second, becomes? [Student] Phrygian. [Tutor] 
Phrygian, that's right! Which is a very nice scale, I'm fond 
of it. 

[Teacher] What is romantic for you? Let's engage in this 
kind of discussion. What is romantic? It's important. 
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Performing 

Directing 


creative ideas 


Students performing a 
prepared piece 
Tutors verbally guide a 
student performance in real 
time 


What is romantic for you? [Students]: To express your 
emotions, along with that establishing a connection. 
[Tutor] Don't you think that being romantic also 
sometimes can mean trying to be a bit more individual 
than you normally are in the real world, to be more 
special? 

A musical activity 

[Teacher] Top string this time. Take that off so you're 
playing - you want that note. There's G. Put your little 
finger back. G7. Put your finger back. The difference 
where your first finger is, yes, that's suspended, that's G. 
You can hear it. 


3.2 Teaching Workflows 


Through our lesson analysis, we were able to identify lesson archetypes which appeared several times in the 
observations. We call these archetypes ‘teaching patterns’, after Eckstein et al. (Eckstein and Bergin, 2002). 
A complete description of the teaching patterns is beyond the scope of this paper but a single example 
flowchart representing a lesson where a student performs in front of their tutor and peers can be seen in 
Figure 2. 



Figure 2. A flowchart describing a peer feedback lesson where a student performs in front of their peers and tutor then 

receives feedback 


3.3 Feeding Back about Music 

Perhaps unlike some other subjects, there is a rather discrete and finite ontology underlying the types of 
feedback one might receive about playing a musical instrument. As part of our analysis, and based on 
previous work, we have developed a detailed ontology to describe feedback on musical performance, shown 
in figure 3. It should be noted that we have identified two broad types of feedback: firstly, feedback 
connected to desirable traits in a musical performance, as shown in the majority of figure 3 and secondly, 
‘information for guiding tactics and strategies that process the domain specific information’ after Butler and 
Winne (Butler and Winne, 1995). The latter might also be expressed as encouraging the learner to develop 
their self reflective skills, their inner teacher. 
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Understanding the difference between 
one’s performance and the perfor- 
mance model aimed at 

Identifying the various compo- 
nents (technical, expressive, 
stylistic etc.) that are creating a 
gap between one's perfor- 
mance and the performance 
model aimed for. 

Understanding the learning 
strategies that can help one 
to close the gap 

Understanding how one can apply 
learning strategies effectively. 

Reproducing or interpreting a 
performance model in a 
critical and/or original way 

Understanding why one's 
reproduction/interpretation of 
a model sounds/feels differ- 
ent from ones previous 
performance. 

Internalising the model 


Encouraging the 'inner teacher' 


Technique 


Feedback 


Ease 



Independence 

Muscle co-ordination and / or support 

Movement 

Dexterity 


Strength 


Overall posture 


Note (intonation, clarity) 
Fingering (for 
instrumentalists) 

Diction (for singers) 

Articulation (legato/stoccato/ etc) 
Accuracy Expressive markings (score 
related: dynamics/timbre, 
tempo changes) 

Rhythm 

Pulse 

Metre 

Timbre control 

Sound 

Dynamic range / control 


stage presence and stage persona. 

Physical response to the 
music performed 

Effectiveness of musical/artistic 

choices [see expressiveness & Presentation 

stylistic awareness] 

Communication with the 
audience (eg. eye contact 
when facing the audience). 


Timbre 
Dynamic range 
Textures 
Articulation 
Tempo changes 


Stylistic awareness 
(choice/control of...) 


choice of timbre/textures 
Shaping of music phrases and 
sections (groups of phrases) through 
timbral/dynamic changes & sense of 
Sound direction 

Balance of musical parts (in 
polyphonic instruments and / 

Expressiveness / ere- or ensemble music) 

ativity (linked to formal / Musical cohesion 

harmonic features) Choice of speed 

Drive 
Groove 

Tempo Sense of breathing 

Flexibility (e.g. accelerando, 
ritardando/rallentando and similar) 
Tenuto and similar 


Figure 3. The ontology we developed which describes feedback about musical instrument playing 

3.4 An Understanding of Problems with Feedback Provision Motivating 
Essential Platform Requirements 

We now have a clear idea of the context within which feedback is given (e.g. lesson flowchart in figure 2) 
and the expected content of that feedback (i.e. the ontology in figure 3). However, we were able to identify 
several reasons why feedback might not be effective, listed below. Note that at this point, we begin to 
consider the basic requirements for our platform which will allow it to address these problems directly. 

(1) The underlying ontology driving the feedback is not well understood. The platform should be able to 
gradually expose an ontology in a range of ways. (e.g. through suggestion of relevant terms, and the 
provision of automated, high level annotations) 

(2) The feedback is not remembered. The platform should make feedback easily accessible for later 
reflection, not hidden away in a forum somewhere, for example. 

(3) The tutor is the sole source of trusted feedback. The platform should embody a community of learners 
pedagogy, to emphasise the value of feedback from peers and tutors alike. 

(4) The feedback given to peers is not honest, e.g. ‘ too nice’. By building a platform that enables more 
precise feedback related to a specific ontology, feedback should naturally become more honest, as the 
emphasis for the feedback is aimed away from the individual and towards particular aspects of a 
performance. 

(5) The relevance of feedback to a particular performance is not understood. The platform should encourage 
the provision of feedback which is specific and well justified. 

(6) The feedback is too narrow. Here, the feedback focuses on a limited part of the ontology, typically due to 
time constraints in a lesson. The platform should encourage a community discussion around a greater number 
of performance aspects. 
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3.5 Platform Design 


The final phase of our methodology was the iterated development of the platform. This process is ongoing, 
but it moved through 8 versions during the first year, where increasing numbers of users were involved at 
each stage. The resulting platform is essentially a repository for audio and video recordings which allows the 
user to upload media then to share it with communities of other users who can then leave comments along a 
timeline. Its key features are listed in table 2 and shown in figures 4 and 5. At the end of this first year of 
development, the system was in active use within 5 undergraduate modules at 2 institutions. In the following 
passage, the key features and motivations for their inclusion will be discussed. 


Music 

Circle 


Tracks cnn Lead sheets Profile Communities Users Help Feedback Logout 


Social Machines Band First Jam G 



Share with community $ 

f* 


c 



Dave Murray, 9 days ago Sedit * *dele 

Everyone's being very tentative and restrained here..AnAnd kind of waiting for someone to bring in 
a pulse... 



Mark, 9 days ago 

You could cut the tension with a knife!! 


*delete 


yeeking, 1 minutes ago 


Reminds me of these guys: 


Bavin - live in London 2005 

< 





► 4) 0:00/3:29 


^delete 


Figure 4. The music circle media discussion interface. 1) The waveform display, showing a highlighted region, 2) The 
tagging dialogue, showing a drop down list of pre-used tags 3) The social timeline, showing sets of time linked comments 
created by several users 4) a discussion thread based on a single region in the recording, including an embedded youtube 

video 
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Table 2. The key features of the system compared to some pre-existing systems which we have used for teaching at our 

institution 


Feature 

MusicCircle 

SoundCloud 

Mahara 

Easy access, personal media repository with mobile media 
capture client 

X 

X 


Simple, transparent sharing 
and community model 

X 


X 

Intuitive discussion interface with content prompting 

X 

X 


Social timeline with region selection 

X 



Powerful discussion system 

X 


X 

Automatic feedback agent 

X 



Suitability for use as a research platform (data access, privacy 
etc.) 

X 


X 


Easy access , personal media repository with mobile media capture client 

The aim is to remove barriers to content uploading and sharing and to make content easily accessible for 
later review. The platform includes simple record and upload apps for iOS and Android to make content 
addition as easy as possible as we identified that the often over-complex process of putting content into VLEs 
can be a serious barrier to uptake for students. 

Simple , transparent sharing and community model 

The aim is to increase user confidence in uploading and sharing media. The platform provides a very 
clear method of controlling who the content is shared with. Also, users can delete any comments made about 
their content. 

Intuitive discussion interface with content prompting 

This feature aims to motivate commenting activity and to encourage use and understanding of appropriate 
terms from the ontology. 

Social timeline with region selection 

Feedback is always connected to a particular range of time in the media. Also, all commenting users have 
individual timelines displayed below the media. This promotes awareness of the community opinions, 
making feedback specific to a person and a time. 

Powerful discussion system 

Users can reply with audio, video, text and so on. Audio and video responses within the platform can then 
become a subject for discussion in themselves, with their own social timeline. 

Automatic feedback agent 

We are developing software agents which are able to feed- back automatically about musical 
performances. They work by comparing different perfor- mances and making high level comments about the 
variations, connected to the feedback ontology. This provides a ‘neutral’ source of feedback and exposes the 
learner to the on- tology. A full description of the feedback agent is beyond the scope of this paper but it is 
built around machine learning and audio analysis techniques. 

Suitability for use as a research platform (data access, privacy etc.) 

We need to be able to ensure the data is appropriately protected and that it can be accessed for analysis 
throughout the project. Also, we need to be able to rapidly prototype and integrate different components to 
our platform for experimentation. 
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© Comment ^ Tag || Attach file to track Share with community 


00:02:87 



anonymous, 1 39 days ago 
Irregular pulse. 


audio feedback X 


©edit X delete 


CS reply 


Figure 5. The social timeline, showing sets of annotations from two users. Each block in the timelines represents an 
annotation connected to a specific region in the recording. Here, the top timeline was created automatically by the 

feedback agent 

The key features of the platform are listed in table 2, where we also compare them to the closest 
equivalent commercial system, SoundCloud and a well known open source e-learning tool with social 
features, Mahara. 


4. A DESCRIPTION OF OUR ONGOING EVALUATION WITH 400 
USERS ACROSS 2 INSTITUTIONS 

The participatory design process aims to suggest then optimise platform features. In a sense, this represents 
an ongoing, evaluation and improvement cycle. However, as stated in the introduction we are interested in 
the evaluation of social machines and the activities they enable at a higher level than basic platform features. 
In this regard, we are running significant case studies with our platform with approximately 400 users spread 
across 2 institutions and 5 different modules. The evaluation scheme consists of qualitative and quantitative 
methods. In particular, we will be using interviews, survey tools and user activity metrics including social 
network analysis. This will allow us to address the research questions listed in section 1.3 with a variety of 
perspectives. We anticipate being able to analyse a data set containing hundreds of media items, thousands of 
comments and many thousands of interactions. 


5. CONCLUSION 

Work has been presented which faces the challenges of how to support educational, social interactions, how 
to deliver personalised music tuition, and how to enable effective feedback. A methodology for addressing 
the challenges has been described which takes real observational data and analyses it into formalisations of 
teaching and learning activities. The outputs from this methodology have been presented, including a list of 
key teaching and learning activities, a flowchart describing the interactions within a typical lesson, and an 
ontology of types of feedback. It has been shown how the outputs have been iteratively interpreted into the 
design for a novel e-learning platform driven by social interactions and effective feedback. The current 
system has been introduced and the ongoing evaluation with 400 users has been described. The immediate 
targets for our future work are to increase the number of learners operating within the platform, to conduct an 
investigation of the wider applicability of the system, for example as a means to deliver recordings of lectures 
and the development of our tool kit for quantitative evaluation of the system. Inspired by the examination of 
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the importance of feedback presented here, the longer term goal is to develop a deeper understanding of the 
nature and importance of feedback in the learning and creative processes. 
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